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Abstract. The experiments described in this paper are part of a larger joint MIT/NASA research effort that focuses on the 
development of a methodology for designing and evaluating integrated interfaces for highly dexterous and multi-functional 
telerobots. Specifically, a telerobotic workstation is being designed for an Extravehicular Activity (EVA) anthropomorphic 
space station telerobot. Previous researchers have designed telerobotic workstations based upon performance of discrete 
subsets of tasks (for example, peg-in-hole, tracking, etc.) without regard for transitions that operators go through between 
tasks performed sequentially in the context of larger integrated tasks. The exploratory research experiments presented here 
took an integrated approach and assessed how subjects operating a full-immersion telerobot perform during the transitions 
between sub-tasks of two common EVA tasks. Preliminary results show that up to 30% of total task time is spent gaining 
and maintaining Situation Awareness (SA) of their task space and environment during transitions. Although task 
performance improves over the two trial days, the percentage of time spent on SA remains the same. This method identifies 
areas where workstation displays and feedback mechanisms are most needed to increase operator performance and decrease 
operator workload - areas that previous research methods have not been able to address. 


INTRODUCTION 

Research is currently being conducted to design and test an intuitive and synthesized telerobotic workstation 
configuration for controlling a high degree of freedom dexterous manipulator for use on the International Space 
Station. The construction and maintenance of the International Space Station is expected to increase the 
number of Extravehicular Activity (EVA) hours by a factor of four over current Space Shuttle missions, 
resulting in higher demands on the EVA crewmembers and EVA crew systems. One approach to utilizing EVA 
resources more effectively while increasing crew safety and efficiency is to perform routine and high-risk EVA 
tasks telerobotically. In response, NASA’s Johnson Space Center (JSC) is developing an anthropomorphic 
telerobot called Robonaut (see Figure 1) that is capable of performing all of the tasks required of an EVA suited 
crewmember. Robonaut is comparable in size to a suited crewmember and requires the operator to command 
over 46 degrees of freedom while performing full immersion telerobotic tasks. The desire to develop a 
methodology for designing integrated workstations is motivated by next-generation robots such as Robonaut. 
The current robotic workstation for the Space Station robots consists of flat panel displays and 6 DOF hand 
controllers. This is insufficient for controlling highly dexterous anthropomorphic manipulators such as 
Robonaut (http://vesuvius.jsc.nasa.gov/er_er/html/robonaut/robonaut.html). The workstation must be designed 
to allow an operator to intuitively control numerous degrees of freedom simultaneously, in varying levels of 
supervisory control, for all types of EVA tasks. Great amounts of research have been conducted in human 
factors areas such as telerobotic interfaces, human-machine interactions, and sensory substitution. However, 
many of the tasks performed in the experiments described in the literature do not capture the variety and 
complexity of the tasks required of an EVA crewmember. In most studies, optimal workstation components are 
determined based on performance of discretized subtasks (such as peg-in-hole, tracking, target acquisition, etc.) 
without regard to the transitions that the operator must go through between tasks performed sequentially 
(Burdea, 1996; Cannon and Thomas, 1997; Massimino, 1992; Sheridan, 1994). In addition, much of the 
research focuses on a particular hardware or software aspect of the workstation without addressing the synthesis 
of components required to tackle the human factors and controls issues of the system as a whole (Kazerooni and 
Snyder, 1995; Liu and Tharp, 1993; Massimino, 1988; Patrick, 1990; Shattuck, 1994; Sheridan, 1993; Vidov, 
1993). Finally, the few groups that have looked at workstations as a whole either have not had to control as 


many degrees-of-freedom as Robonaut demands, or have controlled high degree-of-freedom robots that lack the 
dexterity of Robonaut, and therefore employ hand controllers (Akin, 1986; Homan and Gott, 1996; Li and Cox, 
1995; Sheridan, 1992; Tachi, 1991). The hypothesis of this thesis research is that the most effective design for a 
telerobotic workstation can only be discovered by examining an integrated system, rather than combining 
individually designed components. 



FIGURE 1. Robonaut (in development). 


EXPERIMENT DESCRIPTION 


In an effort to gain an understanding of subject performance during full-length telerobotic EVA tasks, 
exploratory experiments were performed at JSC on the Robonaut predecessor DART (Dexterous 
Anthropomorphic Robotic Testbed). Robonaut was not utilized in these experiments since its construction was 
not completed. DART is similar in design and complexity to Robonaut, however its technology is not as 
advanced, and its dimensions are larger than that of a suited crewmember. Subjects wore a helmet-mounted 
display (HMD) with stereo vision provided by the two DART eye cameras. The subjects' motions were tracked 
using CyberGloves™ worn on the hands and Polhemus™ sensors on the wrists and head [see Figure 2]. Four 
experienced DART operators performed two modified EVA tasks over two trial days: an Orbital Replacement 
Unit (ORU) changeout, and a tool exchange. These two tasks were chosen such that the performance times were 
similar in length, but the content, difficulty and workload were different. The two tasks required the subjects to 
perform operations such as tool grasping and manipulation, rotating the robot about its base, opening and 
closing hinged and sliding doors, target acquisition and tool alignment, manipulating pins and latches, and tool 
stow and un-stow maneuvers. Video data was taken during the trials and DART joint angle data from the base, 
shoulders, elbows, and wrists were recorded. A subjective questionnaire was administered after the last trial 
day. 



FIGURE 2. Subject performing a telerobotic experiment using DART. The subject is wearing an HMD, 
CyberGloves™ , and Polhemus™ wrist and head trackers. 


PRELIMINARY RESULTS 

Performance times for each segment of the tasks were recorded. The average completion times for the ORU 
changeout and the tool exchange tasks were 861 sec and 750 sec respectively. Subjects would scan the area 
between movements or sub-tasks for a greater sense of the workspace layout and their position within it, and to 
decide how best to perform the next task. Up to thirty percent of the total task time was spent gaining better 


sec 


situation awareness (SA) and the average across all subjects and days was 10%. This average percentage 
incorporates time where the subject was solely attempting to increase SA; it does not signify that only 10% of 
the task time was utilized to increase SA. Subjects often swept their gaze around the workstation to learn its 
layout while turning simultaneously, and therefore those times could not be isolated for data analysis. An 
interesting result of the t-test analysis showed that although task time decreased significantly from day 1 to day 
2 (p<0.001), the percentage of time spent on SA remained the same on both days (10.44% and 10.85% 
respectively) (see Figure 3). Additionally, the standard deviations of these percentages were likewise similar 
(9.38 and 8.39, respectively). This indicates that although a subject can increase their performance on a given 
task, there is a constant amount of time the subject must devote to obtaining and maintaining SA. Designing 
displays and feedback mechanisms to aid subjects during these task transitions could further reduce task times. 
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FIGURE 3. Total task time (for both tasks) versus day, and percentage of time spent on gaining and maintaining SA (for 
both tasks) versus day. 


INTEGRATED WORKSTATION EXPERIMENTS 

These preliminary findings are encouraging as they reveal two important observations, 1) as subject experience 
increases SA time required does not decrease without bound, there exists a baseline amount of SA time required 
for a given task and workstation configuration; 2) this baseline amount of SA time may be reduced by designing 
appropriate workstation interface aids. These results support the need to further examine task transitions. In 
response, a set of experiments has been devised that will completely characterize the effect of telepresence 
hardware, sensory feedback degradation and task integration on full-immersion telerobotic task performance 
and workstation design. 

These experiments will be performed at JSC using Robonaut beginning in the winter of 2000. An investigation 
of EVA activities was made to determine the types of actions performed by astronauts during EVA tasks. 
Every movement carried out during an EVA task can be broken into three "basis" movements: 
grasping/releasing, fine position movements and gross position movements. 

Grasping refers to the acquisition or releasing of an object through contact with the hand. For Robonaut, it 
requires fine motor coordination of the hand in, at present, 12 DOF. Since the human arm is redundant, there 
are an infinite number of arm positions that can lead to the same end effector, or hand position. Likewise, the 
grip chosen to acquire the object may vary from person to person; therefore there is no unique grasping 
solution. There may be solutions or grasps that are more desired than others from a stability and controllability 
standpoint, however knowing which grip is optimal requires knowledge of the objects’ handling qualities a 
priori. It is reasonable to expect that operators of Robonaut will be sufficiently trained in all handling qualities 
of the tools they will utilize. Moving involves displacing the arm from an initial location to another desired 
location through a series of torso, shoulder, elbow and wrist joint translations and rotations. The hand may or 
may not be in possession of an object during this arm movement. A further distinction can be made between 
gross movements (such as reaching for a remote object) and fine movements (such as aligning, adjusting and 
inserting objects in real-time). 


Fine positioning is involved in many tasks including tracking, peg-in-hole and target acquisition. Typically for 
these motions, the arm joints will sweep through no more than 30 degrees. The reduced depth perception 
resulting from HMDs employed in telerobotic operations degrades the performance of fine position tasks. 
Gross positioning involves larger motions (greater than a 30 degree sweep through the joint angle) generally 
associated with transporting objects from one location to another, reaching for objects, and turning or 
translating within the worksite. With the FOV limitations (approximately 65 degrees) of the current HMDs, 
collision avoidance becomes an important factor during gross position movements. 

Contrary to previous researchers' methodology of workstation design, it is not efficient to discover the optimal 
workstation or display solution for every subtask (basis movement), since that solution may not be applicable to 
the integrated task. A subject could perform repeated basis movements independently (grasping a tool, aligning 
the arm, and translating the arm), or in combination within a larger EVA task (grabbing a tool, bringing it to a 
toolbox and inserting it). Adding contextual information to the task requires the subject to make decisions on 
the best way to grab the tool given that it is to be subsequently inserted, and must maintain awareness of their 
environment while the tool is transported. This will ultimately increase task times when these transitions 
between basis movements are introduced, and bring attention to problem areas for the operator (decreased SA, 
higher workload, etc.) not previously seen during individual sub-task tests. Looking at task times for individual 
basis movements, versus integrated contextual tasks, is essentially asking - is the sum of the task parts equal to 
the whole of the integrated task? If not, what are the limitations of the system, what feedback mechanisms can 
be provided to increase operator SA and performance, decrease the number of problem areas, and finally, what 
is the maximum performance that we can achieve? By comparing the parts to the whole, you isolate the 
transitions that occur between basis movements. Roadblocks that arise when the task is integrated are now 
apparent, and can be correlated with subjective ratings. Workstation components can be designed to speak 
directly to those problem areas identified by both the objective and subjective performance ratings. 

Protocol Design 

The experimental methodology developed has been designed to both a) isolate the effects of interest and b) 
minimize the effect of confounding variables. 


Basis Task Testing 

A protocol has been developed which tests task time, workload (both objective and subjective) and accuracy for 
gross position, fine position and grasping tasks performed individually. The tasks will span the workspace of 
the robot arm in all directions. Examples include grasping a variety of tools, swinging the arm through a large 
range of motion form an initial to final position, fine positioning of the arm utilizing single joints or multiple 
joints in different orientations (i.e. pointing to a target in different locations). 

In order to describe the baseline performance for each subject and identify their particular subject effects, the 
basis tasks will be performed manually. To reduce the number of variables, and to give the subjects the same 
vision system in all three modalities, the manual tests will be conducted with the addition of the HMD, using 
live video feed cameras placed on the outside of the HMD. In addition to manually, the basis tasks will also be 
performed telerobotically, where the subjects command the robot to go the grasping, fine position and gross 
position tasks. The simulation, however, is limited in fidelity in that there are no contact forces modeled. 
Subjects may command the graphical Robonaut model to move with the same kinematics as the robot, but may 
not test tool handling or other contact force tasks. Therefore, only fine position and gross position basis tasks 
can be tested using the simulation modality. The simulation does have the advantage that the FOV that is 
displayed to the subject can be varied through the simulation software, which allows data to be collected as to 
the effect of FOV on performance. 


Integrated Task Testing 

In addition to the basis task experiments, two representative EVA tasks will be performed that are comprised of 
different numbers and orders of basis movements. These tasks include installing a handrail with captured bolts, 


and a tool exchange between worksite and toolbox. The integrated tasks will be performed in two of the three 
modalities, manually with HMD and telerobotically (fully immersed). Any difference in the performance 
metrics greater than what can be attributed to immersion hardware will be solely a result of the transitions 
between the basis movements. It will then be possible to describe the operator behavior during the transitions, 
and also to determine both objectively and subjectively, where the problems arise (such as highest fatigue, 
workload or task times). 

After determining when SA aids are required (for example, while inserting tools, or during gross motions to 
avoid collisions), displays can be proposed. While an objective indication of the success of a display 
component may be obtained, a subjective evaluation is also necessary. Whereas a visual and auditory display 
can lead to the same objective performance, operators may have a preference for one over the other. It is 
therefore important to identify which aspects of the workstation are in place due to subjective considerations. 

The results of these experiments are expected to guide future full-immersion telerobotic workstation designers 
in providing operators with the most safe, reliable, intuitive and effective robotic interfaces possible. 

ACKNOWLEDGMENTS 

The author would like to thank the Dexterous Robotics Laboratory at Johnson Space Center for use of their 
facilities and personnel, and also for their support of this research effort. 

REFERENCES 

Akin, D. L., "Quantifying human performance in space operations," MIT Report , SSL #23-86, (1986). 

Burdea, G. C., Force and Touch Feedback for Virtual Reality , New York, John Wiley & Sons, Inc, (1996). 

Cannon, D. J. and G. Thomas, “Virtual Tools for Supervisory and Collaborative Control of Robots,” Presence 6(1): 
1-28,(1997). 

Homan, D. J. and C. J. Gott, "An Integrated EVA/RMS Virtual Reality Simulation Including Force Feedback, for 
Astronaut Training", Technical Report AIAA-96-3498-CP, (1996). 

Kazerooni, H. and T. J. Snyder, “Case Study on Haptic Devices: Human-Induced Instability in Powered Hand 
Controllers,” Journal of Guidance. Control and Dynamics 18 ( 1 ): 108-1 13, (1995). 

Li, L. and B. Cox, “Telepresence Control of an Ambidextrous Robot for Space Applications”, Houston, TX, NASA, 

Johnson Space Center Report #JSC-33306, (1995). 

Liu, A. and G. Tharp, “Some of What One Needs to Know About Using Head-Mounted Displays to Improve 
Teleoperator Performance,” IEEE Transactions on Robotics and Automation 9(5): 638-647, (1993). 

Massimino, M. J., Sensory Substitution for Force Feedback in Space Teleoperation. Doctoral thesis in Mechanical 
Engineering, Massachusetts Institute of Technology, Cambridge MA, (1992). 

Massimino, M. J., Effects of force and visual feedback on space teleoperation ; with policy implications. Masters 
thesis in Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, (1988). 

Patrick, N. J. M., Design, Construction and Testing of a Fingertip Tactile Display for Interaction with Virtual and 
Remote Environments , Mechanical Engineering . Cambridge, MA, Massachusetts Institute of Technology: 
109,(1990). 

Shattuck, P. L., “Control of Remotely Operated Manipulation System”, Proceedings of the Advanced Guidance and 
Control Aspects in Robotics Conference, Lisbon, (1994). 

Sheridan, T. B., “Human Factors Considerations for Remote Manipulation”, Proceedings of the Advanced Guidance 
and Control Aspects in Robotics conference, Lisbon, (1994). 

Sheridan, T., “Space Teleoperation Through Time Delay: Review and Prognosis.” IEEE Transactions on Robotics 
and Automation 9f5): 592-606, (1993). 

Sheridan, T. B., Telerobotics, Automation, and Human Supervisory Control a Cambridge, MA, MIT Press, (1992). 

Tachi, S., Tele-Existence and/or Cybernetic Interface Studies in Japan . Human Machine Interfaces for 
Teleoperators and Virtual Environments, NASA Ames Research Center, (1991). 

Vidov, M. E., Visual Interface Issues in a Virtual Environment for Space Teleoperation , Masters thesis in 
Aeronautics and Astronautics . Massachusetts Institute of Technology, Cambridge, MA, (1993). 


